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ABSTRACT 



This paper investigates a solution to support the learning 
of cognitively complex concepts, through the recording, annotation and online 
delivery of multimedia lessons. Recording lessons for online delivery has the 
advantage of utilizing all of the pedagogical skills of the lecturers and 
teachers involved. To facilitate the retrieval of appropriate teaching 
material, the lessons are annotated using a notation called the Flexible 
Structured Coding Language (FSCL) , which facilitates a rich and precise 
description of these concepts, in a natural language-like format. Different 
streams of meta data are produced to describe the lesson's properties, like 
cognitive type and media type. The approach to support the technology-based 
learning of cognitively complex concepts is based on the following elements: 
record complete lessons using the AudioGraph tools; capture different streams 
of meta data and associate those both with and within the media elements; and 
describe the lesson content using the FSCL. The approaches presented are set 
on the context of the IEEE draft standard on Learning Object Meta Data (LOM) . 
(Contains 17 references.) (Author/AEF) 
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TO THE EDUCATIONAL RESOURCES Abstract: This paper investigates a solution to support the learning of cognitively complex 

information center (ERIC) concepts, through the recording, annotation and online delivery of multimedia lessons. 

Recording lessons for online delivery has the advantage of utilising all of the pedagogical skills 
of the lecturers and teachers involved. To facilitate the retrieval of appropriate teaching 
material, the lessons are annotated using a notation called the Flexible Structured Coding 
Language, FSCL, which facilitates a rich and precise description of these concepts, in a natural 
language-like format. Different streams of meta data are produced to describe the lesson’s 
properties, like cognitive type and media type. The approaches presented are set in the context 
of the IEEE draft standard on Learning Object Meta Data, LOM. 
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1 Introduction 

VO The TILE project at Massey University is developing a “Technology Integrated Learning Environment” for the 

^ on-line delivery of education and training material, in a flexible and pedagogically effective manner. The 

Q architecture of this learning environment is described in another paper submitted to this conference (Gehne et al., 

^ 2000) and is not discussed in detail here. One component of this project however, is the provision of on-line 

lectures, which have been recorded by the educators in a multimedia format, using the AudioGraph tools 
(http://www.nzedsoft.com/). These tools have been developed over the last five years (Jesshope et al., 1998) and 
have been used extensively in the on-line delivery of both internal and extramural university teaching (Jesshope, 
1999 and 2000a). There has also been an evaluation of the use of these tools by an independent researcher 
(Segal, 1997). The TILE project is concerned with the flexible and adaptive delivery of this media. We describe 
here, one approach to organising the learning material by annotating it with meta data, which will be queried, 
either explicitly by the student or implicitly through the knowledge that the system holds about a student, in 
order to optimise the student’s learning processes. 

A characteristic of these online teaching resources is that they deal with the presentation of cognitively complex 
concepts and are not restricted to the transfer of simple procedural knowledge. The cognitive sciences have not 
yet provided the pedagogical task analysis necessary to structure such computer-based teaching resources 
(McArthur, 2000). We must therefore, rely on lecturers and teachers to deliver the teaching material for 
cognitively complex concepts. We can not, as is done for simple procedural knowledge, construct a lesson from 
single, basic components but, quite rightly, allow lecturers to record complete lessons. In order to provide a 
collection of teaching material, which is easily accessible to both learners and teachers and which is searchable 
and customisable, we need to provide effective mechanisms to discriminate the various components of these 
lessons. 

The IEEE Learning Technology Standards Committee, working group PI 484. 12, has developed a draft standard 
for Learning Objects Meta data (LOM, 2000), specifying both its syntax and semantics for use in technology- 
supported learning. We use this standard as a framework for the discussion of mechanisms for content 
description and of the units of description, referred to as learning objects in the LOM standard. From there we 
follow on by presenting our own approaches to content description of learning objects. We conclude the paper 
with a description of further work. 
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2 Content Description Mechanisms 

The LOM standard addresses a wide range of issues relating to technology-supported learning applications like 
security, privacy, commerce and evaluation. The purposes of the standard include: ‘to enable learners or 



2 

BEST COPY AYAILAML® 

Page 714 



instructors to search, evaluate, acquire, and use learning objects’, ‘to enable sharing and exchanging of learning 
objects across any technology- supported learning system’, ‘to enable developing learning objects that can be 
combined and decomposed in meaningful ways’, and ‘to enable computer agents to automatically and 
dynamically compose personalised lessons for an individual learner’. The entries in the standard’s base scheme 
for meta data relate to these objectives. For example: 

• Interactivity type (entry number 5.1 in the base scheme) allows distinction between ‘active’ (information 
flows from learner to resource) and ‘expositive’ (information goes from resource to learner) documents. 

• Learning resource type (5.2) describes the cognitive type of the resource, e.g., exercise, experiment or self- 
assessment. This is similar to the ‘cognitive media roles’ as defined by Ram et al (1998). 

• Description (1.5), keywords (1.6), coverage (1.7), and classification (9 with various subheadings) all relate 
to the description of the content of the learning resource. 

The LOM standard gives suggestions for the types of the data elements in the base scheme. The data values for 
entries describing the interactivity type should be taken from a set of restricted vocabulary (‘active’, ‘expositive’, 
‘mixed’, ‘undefined’). For the learning resource type an open vocabulary of single, descriptive terms is 
suggested (‘exercise’, ‘simulation’, ...). The LOM standard refers to future work on a common representation of 
data and suggests, in the meantime, the use of formats such as XML to structure the data. 

In the context of our TILE project we are interested in the detailed description of the content of a learning 
resource in a computer-accessible format. We now evaluate what the LOM standard and other approaches 
provide in terms of content description. 

• The LOM standard gives us a framework for the description of very useful information in relationship to the 
content of the learning resource. Information like interactivity type, resource type, semantic density of 
information (5.4) or difficulty (5.8) facilitates the selection of appropriate material. Being keyword based, 
this type of information is easily accessible. 

• One type of content description within the LOM standard is the classification approach (9) using taxonomies 
(9.2.2) or keywords (9.4). While this provides a high-level description of the content of a learning resource 
and facilitates a contextual relationship between resources, this does not provide a detailed description of 
content in the sense that we are interested in (which is the description of cognitive processes, interactions, 
and interdependencies). 

• Another form of content description within the LOM standard is a textual description (description, 9.3), 
which, being free-form text, has all the power of a natural language to describe the contents. The drawback 
here is that the automated extraction of meaning from this textual description is not possible (Heinrich, 
1999). A search for keywords only within textual descriptions leads to unsatisfactory results (Davenport, 
1996) and the amount of background knowledge necessary to provide the right context for understanding 
natural language is prohibitive (Sowa, 2000). 

• Ram et al. (1999) suggest the Procedural Mark-up Language, PML, which ‘uses cognitive media roles to 
flexibly specify the knowledge structures, the underlying physical media, and the relationships between 
them’. Being based on XML, PML uses tags for the mark-up of information. Within the tag structure, the 
actual descriptions are given in free-form text format. This means, that for our purpose of content 
description, we face the same issues as with the LOM standard. Tag or keyword based information is easily 
accessible but not expressive enough; free-form text format is expressive enough to describe complex 
contents but not accessible enough for automated processing. 

After this discussion on content description, we now want to address the issue of learning objects and in 
particular the application of content description to learning objects in the TILE project. 

3 Learning Objects 

The LOM standard defines a learning object as ‘any entity, digital or non-digital, that can be used, re-used or 
referenced during technology-supported learning’. The base scheme of the standard gives the aggregation level 
(entry 1.9) of a learning object. It specifies that a learning object can vary in size from a single piece of raw 
media to a collection of HTML pages up to a complete course. The descriptions facilitated by the LOM standard 
are applied to the learning objects. There is no possibility to link a description to a specific position or section 
within a learning object. 

In the PML approach by Ran et al. (1999), a PML document contains media tags, which are references to 
external files. These files can have various formats like video, audio or text. Via their media tags the external 
files are linked to knowledge nodes. As the authors write, they are working in domains, which are concerned 
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with the description of procedural knowledge. Therefore the external files will mostly display single objects (like 
a faucet, as in the example in the PML article) or simple procedures (‘re-light the pilot flame’). The approach is 
to make the single multimedia objects very small and to link them into a hypertext network. 

McArthur et al. (2000) write, in the context of interactive learning environments, that the focus will shift from 
teaching procedural knowledge to systems which can provide assistance in learning cognitively complex tasks. 
To explain such tasks we need a longer, more coherent explanation, not just single pictures and brief procedures. 
Hence our desire to capture complete lessons from the teachers and lecturers and to deliver this material within 
an appropriate framework that allows the searching for, or the adaptive presentation of, its component parts. This 
discrimination should be based on the semantic composition of these lessons. For if we cannot split the 
presentations into their atomic parts, we cannot combine them automatically into a pedagogically valuable 
whole, targeted to a specific learner with a particular learning style or ability. We therefore need to make use of 
the pedagogic skills of human teachers in capturing complete lessons and then provide a mechanism for 
describing the composition of the elements inside of these units. 

If we look at these cognitively complex tasks it seems that the current approaches given by LOM and PML are 
not sufficient. PML requires splitting a multimedia presentation into very small parts, which can not provide a 
coherent argument to explain cognitively complex procedures. LOM allows large (and therefore possibly more 
comprehensive) learning objects but facilitates a description of these only at object level and not within the 
object. 

4 Our Approach 

Our approach to support the technology-supported learning of cognitively complex concepts is based on the 
following elements: 

• Record complete lessons using the AudioGraph tools; 

• Capture different streams of meta data and associate these both with and within the media elements; 

• Describe the lesson content using the Flexible Structured Coding Language, FSCL (Heinrich and Kemp, 
1999). 

4.1 AudioGraph Lessons 

The AudioGraph is a tool, which brings multimedia technology to the educators rather than having the educators 
direct the multimedia production through media experts. The direct nature and ease of use of the tools allow the 
educators to capture both procedural knowledge as well as cognitively complex concepts. The tools use 
synchronised images, text and handwriting with direct capture of sound bytes (voice). The presentations 
therefore, have the presence of lectures, or face-to-face communication, and hence the ability to capture whole 
lessons instead of combining single multimedia data elements on web pages. The process of recording a 
presentation is similar to that of presenting in a face-to-face mode, except that the user presents to the computer 
screen. Once the lessons have been captured, web sites with a simple hierarchical structure may be generated 
automatically. The lecturer may use simple static graphics, such as images or presentation slides, or may animate 
diagrams using vector or image-based graphics, to capture the more complex interactions within the material 
being presented, while the whole time, interleaving this with voice-based descriptions. Examples of AudioGraph 
material delivered using a fixed course structure that does not adapt to student progress or queries can be found 
at (http://www-ist.massey.ac.nz/~cijessho/comp-arch/). AudioGraph material, in fact, has all of the advantages 
of a video presentation but, unlike video, can be full screen and can be downloaded at modem speeds. 

While the use of this material has been successful in university teaching (Jesshope, 2000b), feedback from 
students has shown the need for adaptive or selective presentation of the material. And here, AudioGraphs share 
the disadvantages of video media, namely the difficulty in indexing or semantically capturing the contents of a 
unit of presentation. Thus we need to link our meta data and content descriptions to specific positions or 
segments within the lessons. We want to be able to refer a learner to an exact segment of a lesson, which is 
relevant to the learner’s current needs. 

4.2 Streams of Meta Data 

As suggested in the LOM standard, by PML or by Shikano et al. (1998) we want to capture meta data such as 
cognitive type or media type for our learning objects. Because our learning objects are lessons and we describe 
within these lessons our approach has to be different. As already indicated, when looked at from the outside, a 
lesson can be likened to a video document. If we look inside the lesson, we see that the information that is 
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presented by the lecturer uses different perceptual modalities. At one stage the voice of the lecturer will carry the 
information, at another stage a graphic will explain a concept, at yet another point in the lecture, textual 
information will be given. If we want to support different learning preferences regarding visual or auditory 
information, we have to record the medium of delivery of information throughout the lesson. It is not enough to 
specify the whole lesson as a video document. 

The situation regarding cognitive types is similar. Each lesson will contain a sequence of cognitive elements, 
arranged by the lecturer and influenced by their didactic style. Thus, in a lesson, a definition and then an 
explanation might follow a teaching example. We must therefore mark the segments representing the different 
cognitive types. This segmentation for cognitive types can be distinct from the segmentation for other aspects 
like the media types. By describing each lesson by their different aspects, such as cognitive type, media type or 
lecturer’s hints to the students, we arrive at parallel streams of meta data. Using these description streams we are 
able to extract lesson segments according to the preferences of learners, like a segment giving an auditory 
presentation of a particular example on a topic regarded as important by the lecturer. 

4.3 Description of Lesson Content 

As discussed earlier, approaches like the LOM standard describe the properties of and not the content of a 
teaching resource. Where they allow the description of content, they do that in form of ffee-form text, which is 
very difficult to process for knowledge extraction. In the TILE project, we work on describing the content of our 
lessons in a computer-accessible format. In the first instance we want to facilitate the retrieval of the material 
presenting the appropriate contents, in longer term we want to explore the answering of learner questions based 
on our content representation (see Heinrich and Kemp, 2000, for our initial ideas). 

Besides the keyword approach, ontologies (see e.g. Decker et al., 1999) and topics maps (2000) are used to 
describe domain knowledge in the context of multimedia documents. We follow a natural language like 
description approach using FSCL (Heinrich et al., 1 999) which gives us a number of advantages: 

• FCSL descriptions are natural language like and can be understood by any human reader. 

• The vocabulary of the FSCL language can be easily extended by the user. 

• FSCL descriptions are based on a formal grammar and are therefore accessible in their structure using well 
understood parsing techniques. 

• FSCL descriptions can be attached to segments of any multimedia document. 

• FSCL facilitates both qualitative type description (as we need it for describing complex content) and 
quantitative type description (as needed for description of streams of meta information). 

In searching for appropriate teaching material for the particular needs of a learner we can combine these FSCL 
content descriptions with the streams of meta data discussed above. 

5 Description Steps 

In the current phase of our project we are working on the description of existing AudioGraph lessons. Using a 
tool like Snapzpro (http://www.snapzpro.com) we are able to convert an AudioGraph lesson into a video 
document and hence use an existing application, PAC (Heinrich et al., 1998), to apply FSCL descriptions to the 
video. 

As already described, our approach to description is based on streams of meta data and on content description. 
To produce the streams of meta data we perform a quantitative style of coding. For each stream we define the 
required vocabulary: 

• Cognitive type: definition, explanation, example, proof, theory, exercise, solution, introduction, summary, 
revision, conclusion; 

• Media type: visual-text, visual-pictorial, auditory-narrative, auditory-other; 

• Lecturer’s hints: important, complimentary, exam-related. 

For each stream we work through the lesson to produce the relevant descriptions. The way in which this coding 
is performed differs. For cognitive types we code in an exhaustive, non-overlapping style. That means we have 
to cover the whole timeline of the video with exactly one type. For media type we also code exhaustively but 
allow overlapping. We describe each point of time in the video with a media type, but this time we are not 
restricted to exactly one media type. We can create overlapping segments, for example, when an audio narrative 
of the lecturer stretches across two segments of textual and pictorial information. The coding for lecturer’s hints 
is non-exhaustive. These hints will be spread across the lesson and there is unlikely to be a hint at every point of 
time in a lesson. 
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The description for content uses a qualitative type of coding. We construct description sentences, which can be 
attached freely to any lesson segment or point of time. The vocabulary for the content description is largely 
domain dependent. Over time we will be able to provide a generic core vocabulary which then will be extended 
by domain specific terms. We are working on a range of issues regarding the content description: 

• What is the best approach to extract the content description, should it be based on the narrative, textual or 
pictorial information or probably a combination of all? Currently we transcribe the lecturer’s narrative and 
base our description on that transcript. 

• Working on the lecturer’s narrative we have to distinguish carefully between content description and meta 
information. A statement like ‘remember that in a recursive data structure there is no predefined size’ 
contains two elements of information. The word ‘remember’ indicates a cognitive type information of 
revision, which we want to separate from the actual lesson content, which is the statement about the size of 
the recursive data structure. 

• The current FSCL allows the formulation of statements like ‘recursive data structures can be implemented 
using recursive calls to a procedural function’. Based on word categories such a statement can be parsed into 
its subject, verb and object components. To present the type of the cognitively complex content that we are 
working with, FSCL has to be extended, for example to recognise conditional statements. 

At this stage the description process is a completely manual one, based on existing lessons. In the longer term we 
want to develop a procedure to facilitate the description. Included in this procedure should be guidelines for how 
to arrive at good content and meta information description. Parts of the description process should be automated, 
for example the AudioGraph document structure knows about media type and audio transcripts are optionally 
used as an aide memoire for recording speech. Eventually therefore the processes of recording an AudioGraph 
lesson and of describing it should be combined and partially automated in order to produce better quality 
descriptions and a considerable saving in time. 



6 Conclusion 

Using AudioGraph to record lessons gives us the advantage of a rapid and easy-to-use mechanism for the 
creation of learning material in cognitively complex areas. We think it is important to rely on the pedagogical 
abilities of lecturers and teachers to present learning material instead of trying to synthesise this material from 
single elements. We employ various description techniques to make our collections of lessons accessible. Two 
important aspects of our approach are that we attach descriptions to specific segments inside our lessons and that 
we describe the content of these lessons in a form amenable to computer analysis. Attaching descriptions to 
segments inside the lessons allows us to refer the learner to the appropriate section within a lesson instead of just 
referring the learner to the lesson as one learning object. Describing the lesson content, not just on a keyword 
basis but in a rich natural language like way, allows us to find the course material which exactly deals with the 
concepts the learner is trying to comprehend. 

7 Status of the Project and Future Work 

A specification and feasibility study for the TILE project client/server architecture has already been completed 
and is described elsewhere in this conference (possibly!). This architecture allows us to build up large collections 
of teaching material, which can be accessed and monitored quite easily by both on-line and off-line users (Gehne 
2000). The work described in this paper will use student models, past history, student preferences and eventually 
direct questions, in order to present the student with the appropriate material contained within the TILE 
educational repository. We are currently working on creating content descriptions and streams of meta data using 
FSCL to facilitate this. One of the next projects steps will be to investigate how the FSCL description (including 
their segmentation information) can be integrated into the LOM framework. 
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